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Abstract 



CN . We introduce a new mean field kinetic model for systems of rational agents 



interacting in a game theoretical framework. This model is inspired from non- 
cooperative anonymous games with a continuum of players and Mean-Field Games. 
The large time behavior of the system is given by a macroscopic closure with a Nash 
equilibrium serving as the local thermodynamic equilibrium. An application of the 
presented theory to a social model (herding behavior) is discussed. 
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1 Introduction 



This paper is concerned with a dynamic model for an ensemble of rational agents or players 
in the game-theoretical sense. Each agent is endowed with two variables: a type variable 
X which describes the state of the agent such as its position in some social or economic 
neighbourhood, its geographical position, etc., and a decision (or control or action or 
strategy) variable Y which describes the degrees of freedom the agent can play with 
in the game. Each agents tries to minimize a cost function (or equivalently maximize a 
utility function) in the presence of the other players in the framework of a non-cooperative, 
anonymous game [T5l I2l] . By contrast to equilibrium theory, we assume that the agents 
are not choosing the Nash equilibrium instantaneously [22] (also known as Cournot-Nash 
equilibria), but rather work towards this goal by choosing the steepest descent direction 
towards the minimum of the cost function in each infinitesimal time step. In addition, 
this action is overlayed by some statistical noise, giving rise to Brownian fluctuations. 
This setup gives rise to a system of stochastic differential equations. 

We are interested in systems of a large number of agents where a continuum description 
can be adopted, in the way of games with a continuum of players [31 [251 121] also known 
as Mean-Field Games [7J [T2j. Indeed, in the situation of anonymous games with a large 
number of players, the construction of a mean field that serves as a mediator for describing 
inter-particle interactions constitutes an excellent approximation. In this kind of models, 
one describes the contribution of each particle to the creation of a mean field, and the 
effect of the mean field on each particle, by conceiving each particle as infinitesimal, i.e. 
by carrying out a kind of limit process on the number N of particles for N — > oo. We 
refer the reader to [I] for a nice introduction to game theory and Mean-Field Game. 
In the present paper we consider such mean field models, i.e. we consider a continuum 
of players which, under the usual molecular chaos assumptions, can be described by an 
effective equation for the probability distribution of single agents in the phase space (x, y) 
consisting of the type and action variables. 

In this large number of agents limit, a kinetic model for the time evolution of this 
probability distribution f(x,y,t) can be written as follows. 

d t f(x, y, t) + V x - [fV(x, y)} - V, • [fV y <f> f (x, y, t)} = dA y f . (1.1) 

The vector valued function V(x, y) is given by the basic dynamics of the system, describing 
how the state x evolves for a given control variable y. The forcing term V y - [/V J/ $/(x, y, t)] 
in equation (11 -ip describes the agent trying to minimize the cost functional $/ by marching 
in the steepest descent direction — Vy<&/ (In the mean field model considered in this paper 
$j will exhibit a functional dependence on the density /.) The Laplacian on the right 
hand side of equation (11. ip is a consequence of the Brownian noise in the system, with 
the diffusion coefficient d corresponding to the variance. Again, by contrast to the usual 
Mean-Field Game models for agent systems [7J [19] where the optimum control is realized 
instantaneously (leading to the solution of a Hamilton - Jacobi - Bellman equation), our 
agents march an infinitesimal step towards the optimum at each time step by taking the 
steepest descent of the cost functional. 

The substance of this paper is to relate the kinetic description given by equation (ll.ip 
to the well-established game theoretical framework. We show that, in the homogeneous 
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case, when the density / is independent of the state x, steady state solutions of equation 
(11. ip correspond to Nash equilibria. Therefore the model considered here consists of 
agents who try to achieve a Nash equilibrium by choosing their controls in the direction 
of steepest descent towards this equilibrium. In the case of potential games [2T], E3], 
when the cost functional $ f can be expressed as the functional derivative of a potential 
functional, we show that the kinetic equation can be expressed as the gradient flow of a free 
energy. Nash equilibria are the critical points of this free energy. Stable Nash equilibria 
are those which correspond to a global minimum of the free energy, the other minima 
corresponding to metastable ones. For special cases, we can prove that the dynamic 
solution of the kinetic equation converges to these stable or metastable Nash equilibrium 
solutions (see section [5]). We note that potential games have originated from congestion 
games aimed at describing congested traffic situations [23] and that we will recover familiar 
models of traffic flow below (see Eq. 11.21 and comments below). Recently, an approach 
similar to that developed here has been applied to pedestrian traffic [U [2] • 

The main goal of the paper is to investigate the inhomogeneous case, when the proba- 
bility density / depends on the both the state (or type) x and control variable y. We aim 
to derive macroscopic dynamic equations in the state variable x only, which constitute 
good approximations to the solution of equation (11.11) at large scales. Indeed, we look at 
the system over time scales which are large compared to the typical time needed by the 
players to act on their control variables. Simultaneously, we suppose that the interactions 
between the players are localized in the state space x. This corresponds to a situation of 
so-called bounded information where the agents only take into account agents which are 
close to themselves in state space x to make their decision, ignoring agents in the far field. 
In the macroscopic dynamics, we focus on scales in state space which are large compared 
to this interaction scale. Over these large time and state space scales, the distribution of 
agents in the control variable y instantaneously realizes the local Nash equilibrium. This 
local Nash equilibrium describes the statistics of agents in control variable y and depends 
on parameters which may vary over the large scale state variable x and time t. Such 
parameters may be e.g. the local number density p(x, t) of agents at given state x and 
time t, or the mean or standard deviation of the local Nash equilibrium distribution. 

The resulting macroscopic equation represents, in the language of kinetic theory, the 
macroscopic closure of the kinetic equation, using the Nash equilibrium distribution as the 
Local Thermodynamic Equilibrium. For instance, the large time evolution of the density 
p(x, t) is of the form 

d t p{x,t) + V x -(up) = , (1.2) 

where p is related to the probability density / by p(x, t) = f f(x, y, t) dy. The macroscopic 
velocity u(x,t) is given by the expectation of the local velocity V(x,y) over the Nash 
equilibrium distribution. In the simplest possible case the macroscopic velocity u can 
be expressed in terms of the macroscopic density p as u = u p , giving a closed (usually) 
hyperbolic conservation law such as in the Lighthill-Whitham-Richards model of traffic 
[20] . However, in many applications the structure of the macroscopic velocity u is more 
complicated, and additional constitutive equations are needed to obtain a closed system 
from equation (jl.2|) . In such an occurrence a case by case study is necessary. We will give 
such an example in section 

This paper is organized as follows: 
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- In Section[2]we define the basic model, consisting of a system of stochastic differential 
equations, and state under what assumptions the solution can be expressed in terms 
of a mean field density for one effective agent. 

- Section [3] is devoted to the analysis of equilibria. For this purpose it is sufficient to 
consider the homogeneous case where the density function / in (11. ip does not depend 
on the state variable x. In this case the equilibrium solution is given as the solution 
of a fixed point problem and we show that this equilibrium solution is actually a 
Nash equilibrium in the game theoretical sense. In the case of potential games, we 
provide a variational structure and Lypounov functional to equation fll.il) . 

- Section H] is concerned with the inhomogeneous case. We consider the macroscopic 
limit in the regime when the control variable y is adjusted on a much faster time 
scale than that of the evolution of the state variable x and when the interactions 
are nearly local in state space. In the limit, this leads to the macroscopic model 
(jl.2p where the macroscopic velocity u has to be computed from the local Nash 
equilibrium. 

- In Section [51 we apply the framework developed so far to a model of social herding 
behavior, where V is an actual velocity in physical space, and the goal of each 
individual is to adjust to the mean velocity of the ensemble. Here, equilibrium 
distributions are given by the Von-Mises-Fischer distribution. This serves as an 
example of a potential game. However, the macroscopic limit equation (jl.2p is 
not well defined unless some additional constitutive relations are used to determine 
the macroscopic velocity u. An other example pertaining to the evolution of the 
distribution of wealth in economic neighborhood can be found in [12]. There are 
many models of social interactions and group formation based on a game theoretical 
approach (see e.g. [IS]). 

- In section [61 a conclusion is given and perspectives are drawn. 

2 A mean-field model of social dynamics 

We consider iV rational agents (or players) moving continuously in a space of social 
configurations X . Each agent labeled j, j G {1, . . . , N} has social configuration Xj(t) G 
X, depending on time t G R+. It controls its state by an action (or decision) variable Yj(t) 
belonging to a space of control variables y. For simplicity, we suppose that X = M. n and 3^ 
is a compact, orientable, connected manifold imbedded in W p with or without boundary 
and endowed with the Riemannian structure induced by MP. Given (Xj(t),Yj(t)), the 
j-th agent moves in configuration space with velocity Vj(t) = V(Xj(t),Yj(t)), where 
V = V(X, Y) is a given function of the configuration and decision variables. To act on 
their decision variables, the agents impose a given force Fj which will be a function 
of his own and the other agents' configuration and decision variables. In addition to 
this force, each agent's decision variables are subject to Brownian noises which model 
uncertainties in the decision process as well as of the influence of the environment. In the 
game-theoretical literature, this is called a game with mixed strategy. Brownian noises 
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for different agents are independent. In order to constrain the dynamics of the decision 
variables to the manifold y, the resulting combination of the force and Brownian noise is 
projected onto the tangent plane T Y (t) to y at this point. The particle dynamics is given 
by the following Stochastic Differential Equation (SDE): 

Xj = ViX^Y^t)), dYj = P TY . (t) o (if + yffidW}), (2.1) 

where the dot indicates the time derivative, Pt v m is the orthogonal projection onto the 
tangent plane to Yj(t), the symbol o refers to the Stratonowich interpretation of the 
SDE, dWf for j G {1, . . . , N} denote N independent Brownian motions in MP and d is 
the diffusion coefficient. Finally if denotes the force acting on the j-th agent which is 
described below. In the case where 3^ is a manifold with boundary, suitable boundary 
conditions must be given. Such boundary conditions will be specified later on in the 
kinetic framework (see Eq. ( 12.81) ) . That ( 12.11) provides a well-defined SDE on X x y 
follows from the theory descibed e.g. in |16j . 

We denote by X(t) = (Xj, . . . , Xy), Y(t) = (Y u . . . ,Y N ) and % = (Yi, . . . , Y^, 
Yj + i, . . . , Yn). We also write Y = (Yj, Yj) by abuse of notation. We assume the existence 
of a cost function <b N (X, Y, t), such that each agent tries to achieve a Nash equilibrium, 
i.e. relaxes its control variable Yj to an equilibrium Yj(X ,Yj,t) such that 

Y?{X,Y j ,t) = a I g^$ N (X,Y j ,Y j ,t), Vj G {l,...,iV}. (2.2) 

Since such a goal cannot be achieved instantaneously, it chooses the steepest descent 
direction, i.e. it acts a force if on itself such that 

F J N (X,Y,t) = -V Yj $ N (X,Y j ,Y J ,t), Vj'G{l,...,iV}. (2.3) 

We now assume that if is globally Lipschitz with respect to all its arguments, so that 
the system (12.11) has global solutions. 

A more sophisticated way to optimize the action or control variables is to use a 
Hamilton- Jacobi-Bellman equation (see e.g. the Mean-Field Game theory of Lasry & 
Lions [19]). We can also easily generalize this setting to a fiber bundle but we will stay 
in the frame of a trivial bundle (i.e. a cartesian produc) for simplicity. 

Now, we introduce the iV-particle empirical distribution function 

1 N 

f N (x,y,t) = — J2s Xj (t)(x) ® 6 Yj (t)(y), 

and regard / as a map from t G R+ to f N (t) G ViX x y), where V(X x y) denotes the 
space of probability measures on X x y. We assume that in the mean-field limit N — > oo 
of the number of players going to infinity, there exists a one-particle distribution function 
/ = f(x,y,t), which maps t G R+ to fit) G Va, c (X x y) where V ac (X x y) is the space 
of probability measures on X x y which are absolutely continuous with respect to the 
Lebesgue measure on X x y (i.e. the measure on X x y induced by the Lebesgue measure 
onR"xr), such that 

f - /, (2-4) 
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in the weak star topology of bounded measures. We assume that a mean- field cost function 
exists. More precisely, we assume that there exists a map V^X x y) — > C 2 (X x y), 
f i — y $/, such that, for all trajectories (Xj(t),Yj(t)) satisfying (12 .4p . we have 

^(^(t),^^),^-^),^^)^)^^^^^),^^)), Vj e{l,...,N}, Vt>0. (2.5) 

Thanks to these assumption, in the limit N — > oo, the one-particle distribution func- 
tion / is a solution of the following Fokker-Planck equation [26J 

d t f + V, • {V{x, y)f) + V y ■ (F f f) = dA y f, (2.6) 

where Fj = Ff(x, y, t) is given by 

F f (x,y,t) = -V y $ m (x,y). (2.7) 

In (I2.6p . the symbol V y - denotes the divergence of tangent vector fields on y, while A y 
is the Laplace-Beltrami operator on y. Below, we will also use for the tangential 
gradient of functions defined on y. We supplement this system with an initial condition 
/(0) = fo- For short, we will write $/(t) = In the case where y is a manifold with 
boundary, we set a zero flux condition on the boundary X x dy, namely: 

fd n $ f + dd n f = 0, on A" x dy, (2.8) 

where d n f(x, y) denotes the normal derivative of / at (x, y) G X x c^y. 



3 The homogeneous configuration case: convergence 
to Nash equilibria 

Here, we assume that the dynamics of the decision variables is independent of the state 
variables and we restrict the system to the decision variables only. In the kinetic-theory 
framework, this would refer to the spatially homogeneous case, where the spatial depen- 
dence is omitted. Then, / becomes a mapping from t £ [0, oo[ to fit) £ V ac (y), where 
F'ady) i s now the space of absolutely continuous probability measures on y. The cost 
function $ becomes a mapping from / £ V ac (y) to $/ £ C 2 (y). Eq. (12.61) is now written: 

dtf = Q(f), Q(f) = V„ • (/ V y $ f + dV y f) , (3.1) 

with initial condition given by fo- We note that we can write the collision operator Q(f) 
as follows: 

W) = V, ■(/V v (*, + din /))=V v .(/V v Ai,), (3.2) 

with 

fi f (y) = ^ f (y) + d]nf(y). (3.3) 
In the case where y has boundary, then the boundary condition (12.81) reduces to 

fd nj 2 f = 0, ondy, (3.4) 
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where d n ^f(y) is the normal derivative of at y £ cO^- For a given function $(y), we 
introduce the Gibbs measure M$(?/) by: 



M sW = * = / exp ( - *£>) 

The definition of Z$ is such that M$ is a probability density, i.e. it satisfies 



/ M*{y)dy=\. 

Now, we can write 

Q(f) = dVy^ f V y (-^-)). 

We have the 

Lemma 3.1 (%) For any sufficiently smooth function f and g on y, we have 
[ Q(f)^—dy = -d[ V y (-{—) ■ V y ( 1 £—)M 9f dy, 



(ii) We have 



Q{f)4- d V = - d I V v(4~) 2 M* f dy<0. 



y&y M<s, f 



(3.5) 



(3.6) 



(3.7) 



(3i 



(3.9) 



Proof, (i) Multiplying f l3.Tj) by g/M<f, f , integrating over y and using Green's formula on 
y, we get (13. 5J) . In the application of Green's formula, the boundary term is either absent 
when 3^ has no boundary or vanishes due to the boundary condition (13.41) in the case 
where y has a boundary. Indeed, we notice that 

J— = Z$ f e^, Q{f) = dZ* / Vy(M* f V y e ! Z). (3.10) 

Therefore, the boundary term in Green's formula is written 

dZ^. [ M<s>f (y)d n (e^)(y)-?-dS(y) = 0, 

Jytdy IV1 <S>f 

where dS(y) is the measure on dy and where the integral is zero because 

. ■ 1 

d n (e i )(^) = ^ e d 9 nVf(y) = Q, 

by virtue of (13.41) . 

(ii) We let g = f in <g£E§ and get (j3T9jh ■ 
From Lemma [3. 1[ we deduce the following 
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Proposition 3.2 The distribution function f G V ac {y) is an equilibrium solutions, i.e. 
a solution of Q(f) = if and only if f is of the form f eq where f eq is a solution of the 
following fixed point problem: 

fe q {y) = ^ exp( — ), Z$ = / exp( — )dy. (3.11) 



Proof. First, suppose that Q(f) = 0. Then, J y€ yQ(f) jj^- dy = 0. Therefore, thanks 
to (13.91) . jj— is a constant. Using the positivity of M$ and its normalization condition 

(13. 6p . we get / = M$ / . Consequently, for / to be an equilibrium, it has to satisfy the fixed 
point problem (13. lip . Conversely, if f eq is a solution of the fixed point problem (13. lip , 
then jj eq = 1 and Q(f e q) = follows. ■ 

'/eq 

We now show that equilibria (13. lip are Nash equilibria for the mean-field game (also 
known as non-cooperative anonymous game with a continuum of players [7J) associated 
to the cost function fif(y). For such a game, a Nash equilibrium measure /ne 6 V(y) is 
such that [7] (see also [3 |6]) there exists a constant .fT and 



(y) = if \/y e Supp(/ : 

^{y)>K Vyey. 



NE y j 



(3.12) 



This definition is equivalent to the following statement [7J: 

/ ^ f ne(v) f^(y) dy = M fi fNE (y) f(y)dy. (3.13) 

Eq. (13.131) is called the 'mean-field' equation. Now, we have the following 

Theorem 3.3 Let f G V ac {y)- Then the two following statements are equivalent: 

(i) f is an equilibrium $3.11\) . 

(ii) f is a Nash equilibrium llS.lfy) . 

Proof, (i) ^> (ii). Let / eq be an equilibrium (13.111) . Since y is compact and $/ is 
continuous on y for any / G Vac(y), then $/ eq is bounded. Therefore, its support is 
the entire manifold y and the second line of (13.121) reduces to the first line. We easily 
compute that K = — dba.Z$ fe . Therefore, / eq is a Nash equilibrium (13.121) . 

(ii) =>■ (i). Let / NE be a Nash equilibrium (13.121) . We show that Supp(/ NE ) = y. Indeed, by 
contradiction, suppose Supp(/ NE ) ^ y. There exists y G y such that f NJS (y) = 0. Then, 
because of the log inside (13.31) and the boundedness of 3>/ NE , we have fif NE (y) = — oo which 
is a contradiction to the second line of (I3.12p . Therefore, by the first line of (13.121) . /ij NE 
is identically constant over the entire space y. From the expression of /i/ NE in (I3.3p . / NE 
is proportional to exp(— $y NE /<i), which means that it is an equilibrium (13. lip . ■ 
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The mean- field model (13. ip . (13. 2p can be recast as a transport equation as follows 



d t f + V y -(vf) = 0, (3.14) 
v = —VyHf. (3.15) 

It describes the bulk motion of agents which move in the direction of the steepest descent 
towards the minimum of \if. When all agents have reached the minimum of /z/, then fif is 
a constant and describes a Nash Equilibrium. Therefore, in the proposed dynamics, the 
agents choose as their action to move in the steepest descent direction towards the Nash 
equilibrium. 

Remark 3.1 Variational structure and potential games |21] . We suppose that 
there exists a functional U(f) such that 

where ^jp- is the functional derivative oflA defined by 

6 -^p-(y) <j>{y) dy = lim -{U{f + s<j>) - U(f)), (3.16) 

for any test function <fi{y). We note that the existence of such a functional gives a very 
strong constraint on $ (if f were finite-dimensional, i.e. if y were replaced by a finite 
set and eq. ( Iff. 1\) by a system of ordinary differential equations, that would mean that $ 
is the gradient of the scalar potential U). We will call U the potential energy. A game 
associated to such a cost function $ is called a potential game JH]/. We now introduce the 
entropy functional: 

S(f) = d f f(y)]nf(y)dy, (3.17) 

and the free energy 

T(f)=U(f)+S(f). (3.18) 

It is a simple matter to find that 

ss(f) 



(y) = d\nf(y). 



Sf 

Therefore, we have 

-(y) = *f(y) + dhif(y)=n f (y), (3.19) 



Sf 

so that, in this setting, /i/(w) can be seen as the 'chemical potential' associated to the free 
energy J-'(f). Now, we can recast k3. h3.2\) as 

dtf = Vy ■ {(Vyfl)f) = Vy • (V y (^^) /) . (3.20) 
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But, for a function f(y,t), we have 



Inserting A3.20\) into the above equation, and using Green's formula, we have: 

| t)) = -f f(y, t) v /^ff'^ y) \y, t) dy : = -V(f(; t)) < 0. 



dt 



Therefore, T is Liapounov functional for this dynamic andT> is the free- energy dissipation 
term. By fine analysis of T>, it is possible in some cases to deduce decay rates from this 
kind of estimate JT^j \2~8$ . Equilibria given by A3.11\) are critical points of J 7 subject to 
the constraint j f dy = 1 and the chemical potential fj, is the Lagrange multiplier of this 
constraint in this optimization problem. Each of these critical points corresponds to a 
Nash equilibrium. However, these critical points are not necessarily global minimizers of 
the free energy. Among these equilibria, the ground states, which are the global minimizers 
of jF are the most stable ones. Other equilibria are either not stable or only locally stable 
(or meta- stable) . The co-existence of several stable equilibria may give rise to phase 
transitions and hysteresis behavior if bifurcation parameters are involved and varied. 

4 The inhomogeneous configuration case: Nash Equi- 
librium macroscopic closure 

Now, we return to the inhomogeneous configuration case (12.61) . (12. 7p where the positions 
of the players in the social configuration space is considered. The goal of this section 
is to investigate the ensemble motion of the players at large time scales, averaging out 
over their individual decision variables. For this purpose, we have to assume a temporal 
scale separation, where individual decisions are fast compared to the evolution of the 
ensemble of players in configuration space. We also need to observe the system as a bulk, 
averaging out the fine details of the individual players in configuration space. Therefore, 
we will introduce a suitable coarse-graining procedure. Taking advantage that at large 
times, individuals relax their decision variables towards that corresponding to a global 
Nash equilibrium given by (13. lip , we use this equilibrium as a prescription for the internal 
decision variable distribution of the agents. In this section, we provide the details of this 
coarse-graining process, known as the hydrodynamic limit in kinetic theory. 

In order to manage the various scales in a proper way, we first change the variables 
to dimensionless ones. Let t be a time unit and let xq = ato, where a is the typical 
magnitude of V. We choose to m such a way that the magnitude of $ is 0(1) and 
introduce the quantity d = dt = 0(1). The decision space y is already dimensionless 
and the variable y does not require any scaling. Introducing new variables x, = x/xq, 
t = t/t , f(x,y,t) = x%f(x,y,t), V(x,y) = V(x,y)/a, $f{x,y) = Eq. (J2J1 is 

written: 

d-tf + V, • (V(x, y)f) + V, • {Fj f) = dAj, (4.1) 
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where Fj is given by 

F f (x,y,t) = -V y $ f ix,y), (4.2) 

We now introduce the macroscopic scale. Refering to the discussion of the beginning 
of this section, we change the configuration space unit and the time unit to new ones x' , 
t' Q which are large compared to Xq, to- Specifically, we let e <C 1 be a small parameter 
and define x' Q = x /e, t' = t /e. Here, e refers to the "small" average change of the 
configuration of the ensemble of agents on the "fast" time-scale of the evolution of the 
decision variables. In the kinetic framework, e would be a measure of the particle mean- 
free path in macroscopic units. By doing so, we change the space and time variables x and 
t to macroscopic variables x = ex, t = et and define f(x,y,t) = e~ n f(x,y,t). Inserting 
this change of variables into (14. ip . (14. 2p . we are led to the following perturbation problem 
(dropping the hats and tildes for simplicity): 

e(d t f £ + V x . • (V(x,y)f)) + V, • (F/ s f) = d\f, (4.3) 

where Ffe is given by 

F e f (x,y,t) = -V y & f {x,y). (4.4) 

For a distribution function f(x,y), f £ V &C {X x y), we assume that $j can be 
developed as follows: 



with 



& f (x,y) = $ p{x) ,„ x (x,y) + 0{e 2 ), (4.5) 



v x (y) = -rwf{x,y), P( x )=l f(x,y)dy, (4.6) 



p{x) 



and is a map [0, oo) x V^iy) — > C 2 (X x y), (p,u) i-)- In short, u x is the 

conditional probability density of / conditionned by fixing the position x £ X and it 
belongs to "P ac (3^)- Eq. ( 14. 5 p states that, up to factors of order 0(e 2 ), the cost function 
is a functional of this conditional probability and of the density only, and therefore, only 
depends on local quantities at social position x. The 0(e 2 ) term collects all non-local 
effects in social position space. These effects are supposed to be much smaller than the 
local ones. This is an expression of the scale separation in social space: local effects in 
social space are supposed to have a much bigger influence that non-local ones on a given 
subject. 

The macroscopic limit is about taking the limit e — > in this set of equations. In 
order to do so, we write (14.31) . (14.41) as follows: 



with Q given by 



d t r+v x -(v(x, y )n = -Q(n, (4.7) 

£ 



Q(f) = V, • (V y * P ( x ,t), u x , t f + dV y f) , (4. 
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and where p(x,t), v X)t are related to f(t) by (14. 6p . Here, we have used (14. 5[) to replace 
$ e by <3> in (J4TTJ) , (14.81) and dropped the remaining 0(e) terms. In the example section 
below, we will show that this assumption is actually quite natural. 

Here again, we emphasize that (x, t) now refers to slow variables. The left-hand side of 
(14. 7p describes how the distribution of agents as a function of the external variables (the 
social configuration x) evolves. This evolution is driven by the the fast, local evolution 
of this distribution as a function of the individual decision variables y described by the 
right-hand side. The parameter e at the denominator highlights that fact that the internal 
decision variables evolve on a faster time-scale than the external social configuration 
variables. The fast evolution of the internal decision variables drives the system towards 
an equilibrium, i.e., solution of Q(f) = 0. Such a solution is referred to in physics as 
a Local Thermodynamical Equilibrium (LTE). Below, we use the results of the previous 
section to show that, in this case, the LTE's are given by Nash equilibria. 

To highlight this fact, by factoring out p{x,t) from the expression of Q in (I4.8p . we 
can recast it as follows: 

Q(f) :=p{x,t)Q piXtt) {v Xit ), (4.9) 

where, for any p G M + , we define the operator Q p acting on V^iy) as follows: 

Q p (u) := Vy-(Vy$ p ,„v + dV y v). (4.10) 

The equation Q(f) = can then be recast (supposing that p(x,t) ^ 0) into Q^x^i^x^) = 
0. But the operator Q p freezes the slow variables (x, t) and acts only on the distribution 
of agents in the decision variable y. Therefore, this equation is merely a homogeneous 
configuration problem and we can apply Proposition 13.21 to solve it. This leads to the 
following lemma whose proof is a direct application of Proposition 13.21 and is omitted. 

Lemma 4.1 The LTE, i.e. the solutions ofQ(f) = are given by 

f(x,y,t) = p(x,t) is eq , P (x,t)(y), (4.11) 

where v e q,p{y) is o solution of Q p (u) = 0. Such solutions v e q,p{y) o,re given by the resolution 
of the Nash equilibrium fixed point problem 

-eUy) = y^— exp ( - ^M), Z W = / exp ( - ^?M) d y. (4.12) 

Now, we can state the result for the coarse-graining limit e — > inside Eq. (14 .7p . We 
have 

Theorem 4.2 Suppose that the solution f £ to (L/.7p converges to a function f when e — > 
smoothly, which means in particular that all derivatives of f £ converge to the corresponding 
derivative of f . Then, formally f is given by an LTE (f^.iJp . The density p(x,t) satisfies 
the following conservation law: 

d t p + V x -(pu) = 0, (4.13) 
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with 

u = u[x, v eq , P (x,t)}, (4.14) 
being the mean velocity of v e q, P (x,t) an d u[x, v] is given by 



u[x 1 v}= V{x,y)u(y)dy, (4.15) 
Jyey 

for all distributions v G V ac {y). 

Proof. From (14.71) . we have that Q{f £ ) = 0(e) and owing to the convergence assumptions 
made on f £ , we have Q{f) = 0. Thanks to Lemma |4.1[ / is of the form (14. lip . Now, 
observe that 1 is a collisional invariant of Q, meaning that J y£ yQ(f)(y) dy = 0, for all 
functions f(y) (simply by Green's formula and the boundary conditions). Therefore, 
integrating (14. 7p with respect to y leads to 

d t p £ + V x ■ (p £ u £ ) = 0, (4.16) 

with u e (x,t) = u[x, i>zt\ where u[x, v] is given by (14.151) and u xt is related to f £ by 
the first eq. (14. 6p . Then, taking the limit e — > in (I4.16P and using that p e — > p and 
v s x t y v e q,p(x,t), w e get that u £ — > u[x, is cq ,p(x,t)} and that the limit of Eq. (I4.16P is precisely 
gll. ■ 



Remark 4.1 We note that Eq. I[4-13\ ) (complemented with an initial condition po{x) and 
possibly boundary conditions) does not necessarily lead to a closed system. We will provide 
examples in the next section where additional equations may be required to provide a closed 
problem. However, in many cases, the solution of the Nash equilibrium problem li3.11\) is 
not known explicitly. This suggests the development of coarse- graining strategies, based 
on e.g. the Heterogeneous Multiscale Method J2J/ or kinetic upscaling 07^/ . 



5 Models of social herding behavior 

5.1 General framework 

Here, we specify the potential §f(x,y) as given by the following kernel: 

<$> f (x,y)= k(x,y : x',y')f(x',y')dx'dy', (5.1) 

J(x',y')exxy 

where (x, y, x', y') G (X x y) 2 \-y k(x, y, x', y') G R is a given function. To be more specific, 
we focus on a model of social herding behavior, where pairs of agents try to minimize the 
angle between their respective social velocities. Namely, we set: 

k(x, y : x', y') = -K(x, x') V(x, y) ■ V(x\ y'), (5.2) 

where V(x,y) is the velocity in social space specified earlier and the dot refers to inner 
product in the vector space X . By trying to minimize the angle between their own velocity 
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and that of their neighbors, the agents adopt a mimetic behavior, and tend to move in 
social space in the same direction as the others. We can write 

$ f {x,y) = -V(x,v)-Wf{x), (5.3) 

with 

W f (x) = [ K(x, x') V(x', y') f(x', y') dx' dy' G X. (5.4) 

We can view W/ as some average of V(x, y) over /. 

Now, let us first focus on the homogeneous configuration case, letting K = 1. In this 
case, f(y,t) satisfies (13. ip with $/(?/) given by: 



*f(y) = -V(y)-W f , W f = V(y')f(y')dy' (5.5) 

Jy'ey 

By (13.1 ip the Nash equilibrium is now depending on a parameter W G X. It is denoted 
by M w and given by (|3.5p with $(y) = —V{y) ■ W, i.e. 



(y) = J- exp [hv{y) ■ W)) with Z w = [ exp {-SV{y) ■ W)) dy. (5.6) 

Now, eq. (13. lip which defines a Nash equilibrium is replaced by a 'compatibility condition' 
deduced from ( 15.4D and which expresses that 

or equivalently 

W / exp ■ W)) dy= [ exp (1(1%) ■ Wj) V(y) dy. (5.7) 

Jyey " Jyey " 

The associated game is a potential game. Indeed, we introduce the potential energy 

W) = -V/| 2 . (5-8) 



According to definition (13.161) . we have, for all test functions <f)(y), 

-(y)<P(y)dy = -W r —/<P(y)dy 



= -w r 


1 


Jy&y 


= -w r 


s 





Sf wirw " J J y ey Sf 

V(y) 4>{y) dy 



^f(y)cj)(y)dy, (5.9) 

yey 

where, to pass from the first to the second line, we have used that, in the homogeneous 
configuration case, W/ is linear with respect to /. So, we get: 

n 



which shows that the game with cost function fif(y) given by ( 13. 3 j) is a potentiel game 
associated to potential U. Its variational structure is associated to the following free 
energy functional (thanks to (13. lTj) ( 13. 18j) and ( 15. lUj) ) : 



Hf) = s{f)- V/l 2 . (s.ii) 



2 

In economics, one may be interested in the 'social cost' defined by 



C(f) = / Mv) f(y) dy = S(f) - / 1> f (y) f(y) dy, (5.12) 
Jyey Jyey 

where the second equality comes from (1 3 . 3 j) . (13.171) . Now, taking = / in (15.91) we have, 

for aii / e v ac (yy- 

<$> f {y)f{y)dy = [ 5 ^fi{y) f{y) dy. 
yey Jyey °J 

But, noting that U(f) is a quadratic function of /, we have the following identity, which 
is valid for all degree 2 homogeneous functions: 

5 -^P-(y)f(y)dy = 2U(f). 
yey °J 

It follows that, for all / G P ac (X): 

/ $ f (y)f(y)dy=-\W f \ 2 . (5.13) 
Jyey 

Then, the social cost (15. 12[) has the expression 

C(f)=S(f)-\m 2 - (5-14) 

Note, that the free energy ( 15.111) and social cost ( 15. 141) differ by a factor 1/2 in front of 
|W/| 2 . This means that they differ except if W/ = (which is a kind of a measure of the 
social disorder). We note that a Nash equilibrium gives rise to a minimizer of the free 
energy instead of the social cost. The reason is that individual players make strategies 
without taking into account the cost of the freely available social infrastructure. To 
correct this discrepancy, one has to make players pay for the use of this infrastructure by 
e.g. assigning taxes. In the present cases, taxes would lead to a cost function equal to 
^}{v) ~ V{y) ' Wf- An example pertaining with city planning can be found in [6]. 



5.2 Example: animal herding model 

In this section, we consider a special case of the above one which describes the herding 
behavior a group of animals or a human crowd. The social space X coincides with the 
geographical space lR n (with n = 2 (for crowds) or n = 3 (for fish schools for instance)). 
The decision variable y is the direction of the motion of the individuals and is such that 
y G y = S n ~ 1 , where S n_1 is the unit sphere of R n endowed with the Lebesgue measure dy 
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(normalized such that the total measure of y is equal to 1). The function V(y) relating 
the decision variable to the physical speed is independent of x and simply given by: 

V(y) = y, 

where the speed (supposed uniform and independent of position) is normalized to 1 
through the non-dimensionalization procedure. We take the same kernel (15 HJ) as before, 
which leads to the cost function 

$f(x,y) = -yW f (x), (5.15) 

with 

W f (x) = [ K{x,x')y' f(x',y')dx' dy'. (5.16) 

In the homogenous configuration case (where we set K — 1), the free energy is still 
given by (15.111) where now W/ is given by 

W/= / y'f(y')dy'. 

Then, from (15. 6p . the Nash equilibrium is given by a so-called Von-Mises-Fischer (VMF) 
distribution 



Mi 



(y) = -^exp (-{y ■ Wj), Z w = [ exp {-{y ■ Wj) dy, (5.17) 

Zj W a Jyesn-i a 

where W G M. n is a solution of (15. 7p . In the present context, this equation is written: 

W [ exp (hy ■ Wj) dy= f exp (-{y ■ W))ydy. (5.18) 

By rotational symmetry and expressing the integrals in (I5.18P in polar coordinates, we 
can write 

W=\W\n, (5.19) 
where Q G is arbitrary. The quantity k — \W\/d satisfies: 

c(«) = d«, (5.20) 

with 

Ce KCOB ° cos6 sin"" 2 d6 
c(k) = 5 , 5.21 

and cos 6 = y ■ Q. Then, the VMF distribution ( I5.17P is more conveniently written M k q 
(where k G M+ is the concentration parameter and Q is the mean direction) as: 

M KSl (y) = ^rexp( K (yn)), (5.22) 

K 
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with Z K given by the denominator of (15.211) . The quantity c(k) is the order parameter. It 
is an increasing function of k which satisfies < c(k) < 1. When c(k) ~ 0, then M K ^(y) 
is nearly isotropic (i.e. M K ^(y) m 1). On the other-hand, when c(k) — > 1, which happens 
when k — y oo, then M^iy) — » 5n(y) (see details in 0, [33]). 

Now, we look at the solutions of the compatibility condition (15.201) . This analysis has 
been performed in [9l [T3J. We only summarize the final results in the following. 

Theorem 5.1 (%) J/c'(0) = \ < d, then, the only solution of Eq. [5~M) is k = 

and the associated equilibrium Ii5. 22\) is the uniform distribution Mq = 1. It is a stable 
equilibrium. 

(ii) If c'(0) = - > d, then, there exist exactly two solutions of Eq. Ii5.20\) : n = and 
another solution denoted by K d > 0. n d is a strictly decreasing function of d e [0, -] onto 
(+oo,0]. The associated equilibria 115 . 22\) are the uniform distribution Mq = 1 associated 
to k = and all VMF distributions M Kd n where Q takes any value on the sphere S n ~ 1 . 
The uniform equilibrium is now unstable and the VMF equilibria M Kd ^ for all Q G S n_1 
are the ground states of the free energy and are stable. 

We refer to [9J [H] for the precise mathematical statement of the stability result, as well 
as for rate estimates of convergence to the equilibria in the homogeneous configuration 
case. 

Remark 5.1 If n = 1, then S™" 1 = {—1,1} and c(n) = tanh(K). The compatibility 
condition Ii5.20\) is the same as the mean-field equation in the Ising spin model for ferro- 
magnetism J77J/. 

Now, we apply the coarse-graining procedure with Nash equilibrium closure developed 
in section H]to this special case. To implement this closure, we first need to verify condition 
(I4.5p . For this purpose, we assume that 



with k(r): r G [0, oo) — > M + a given kernel. The quantities associated to such kernel K 
by (I5.15P and f 1 5 . 1 6 j) are denoted by <M and WS. For simplicity, we assume 



Then, inserting (15.231) into f 1 5 . 1 6 j) and expanding in powers of e, we get, for / G V^X x y): 



K{x,x') = k( 



x — X' 



(5.23) 
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WUx)=p(x)W I/x +0(e 2 ), 



(5.24) 



with for all v G V ac (y), 




(5.25) 



By inserting this expansion into (15. 15ft . we get 

& f (x,y) = $ p ^M + 0(e 2 ) 



(5.26) 
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where, for p G R+ and v G V^iy), we set 

$ p , v (y) = -pyW u . (5.27) 

We note that, in order to recover the homogeneous configuration setting of the beginning 
of this section, W/ must be replaced by pW u in (I5.15p . It follows that the compatibility 
condition (15.201) becomes 

c(hi) = K — , 

p 

and that the Nash equilibrium solutions are now the VMF distributions M Kid/p Q, with 
Q G S 71-1 . The discussion of Theorem 15.11 is still valid provided that d is replaced by 
d/p everywhere. Therefore, there are two regimes corresponding to items (i) and (ii) in 
the statement of Theorem 15.11 We successively describe the models resulting from the 
application of Theorem 14.21 for these two regimes. 

(i) Large noise or small density case: - > K Then, the only Nash equilibrium being 
the isotropic distribution Mq = 1, the macroscopic velocity u as given by (I4.14p . 
flUSD is 

u = u[l] = y dy = 0, 

by antisymmetry. The macroscopic equation (I4.13P reduces to 

d t p = 0. 

In order to get a meaningful macroscopic model, we must rescale time to diffu- 
sive scales. In this diffusion approximation procedure leads to a nonlinear 
diffusion equation for p. Details can be found in [9]. 

(ii) Small noise or large density case: - < -. Then, we use the ground-state Nash 
equilibrium M Kd/p n, where Q G S" -1 . In this case, / is given by (14. lip , i.e.: 

f(x,y,t) = p{x,t)M Kd/ (y). (5.28) 

By the computations above, the mean velocity u is given by 

u(x,t) = c(K d/p )Q(x,t) ^ 0. 

The Nash Equilibrium macroscopic closure equation ( 14.131) gives 

dtp + V, ■ (c(« d / p )/ofi) = 0. (5.29) 

We note that it does not provide an equation for f2(x, t) yet and as such, would lead 
to an ill-posed problem. However, by using the concept of Generalized Collision 
Invariant (GCI) [13], it is possible to derive the equation for Q. This equation 
reads: 

d t n + b(p)(n ■ v x )n + q( p )p q ±v xP = o, (5.30) 
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where b(p) and 0(p) are real- valued functions of p and P n ± = Id — Q ® f2 is the 
orthogonal projection of onto the hyperplane space (Spanjfi})- 1 orthogonal to 
Q. The functions 6 and are not specified here. They are obtained through the 
application of the GCI to (14. lip . Details can be found in [9]. We note that because 
of the presence of the projection Pq± the constraint \Q\ = 1 is propagated in time 
as soon as it is verified at time t = 0. We note that the system is not in conservative 
form but, in some conditions, it can be shown to be well-posed [II] . 

This example illustrates that the Nash Equilibrium closure can be effectively used to 
derive macroscopic closures. However, the mere mass conservation equation (I4.13P may 
be not be enough to provide a well-posed closed system and that additional techniques 
must be called for in order to find a closed system. 

6 Conclusion and perspectives 

In this paper we have provided a framework for the time evolution of a system of rational 
players in a non-cooperative anonymous game with a continuum of players (or Mean-Field 
Game) which collectively make their decision by choosing the steepest descent direction of 
the individual cost functions. Assuming that the individual actions are fast and localized 
in state space, we have derived a macroscopic dynamic which describes the large scale 
evolution of the parameters of the local Nash equilibria. In forthcoming works, we plan 
to apply this framework to various phenomena such as the evolution of the distribution of 
wealth in the economic neighborhood, opinion formation, social dynamics and collective 
decision making. 
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